Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases

摘要

Text representation using neural word embeddings has proven efficacy in manyNLP applications. Recently, a lot of research interest goes beyond wordembeddings by adapting the traditional word embedding models to learn vectorsof multiword expressions (concepts/entities). However, current methods arelimited to textual knowledge bases only (e.g., Wikipedia). In this paper, wepropose a novel approach for learning concept vectors from two large scaleknowledge bases (Wikipedia, and Probase). We adapt the skip-gram model toseamlessly learn from the knowledge in Wikipedia text and Probase conceptgraph. We evaluate our concept embedding models intrinsically on two tasks: 1)analogical reasoning where we achieve a state-of-the-art performance of 91% onsemantic analogies, 2) concept categorization where we achieve astate-of-the-art performance on two benchmark datasets achieving categorizationaccuracy of 100% on one and 98% on the other. Additionally, we present a casestudy to extrinsically evaluate our model on unsupervised argument typeidentification for neural semantic parsing. We demonstrate the competitiveaccuracy of our unsupervised method and its ability to better generalize to outof vocabulary entity mentions compared to the tedious and error prone methodswhich depend on gazetteers and regular expressions.